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In 2010, the National Center for Learning and Civic Engagement (NCLCE) established the Schools of 
Success, a national network of 19 schools that use service-learning as an instructional strategy/ The 
schools were part of a three-year project to examine how the elements of service-learning might 
enhance key student outcomes, such as academic performance and civic engagement. In this brief, we 
report on the relationships observed between students' participation in service-learning and academic 
performance and school attendance. 


Key Findings 

> In several instances, the attendance of students who participated in service- 
learning activities was statistically higher than their peers who did not participate 
in service-learning. 

> Overall, the academic performance of students who participated in service- 
learning was not statistically better than their peers who did not participate in 
service-learning. 

> Our findings show, however, that in some instances, service-learning may help 
improve students' reading performance. This is particularly the case when: 

• Service-learning is high-quality, as determined by students' evaluation of 
their service-learning experience 

• Students' reading performance is in the bottom quartile. 


i^NCLCE 

National Center FOR Learning 
★ ^ AND Civic Engagement 



Background 


The NCLCE selected participating schools for the Schools of Success network based on their support of 
five elements critical to the successful, school-based integration of service-learning. Research suggests 
that student achievement and success through service-learning result from these elements: 

1) Vision and leadership 

2) Curriculum and assessment 

3) Professional development 

4) Community-school partnerships 

5) Continuous improvement.^ 

Each school received funding over three years ($5,000 per year), on- and off-site professional 
development opportunities and ongoing technical assistance to expand and deepen existing service- 
learning initiatives and build greater capacity for service-learning within their school and district. In 
return, the NCLCE asked schools to test and learn from leadership strategies that integrate and sustain 
quality service-learning for all students to succeed in school and their communities. 

The Schools of Success network was funded by the State Farm Companies Foundation and the 
Corporation for National and Community Service/Learn and Serve America. Because both funders 
required their own evaluation, each year the NCLCE completed two separate evaluations of the 
participating schools in the program (see Exhibit 1). State Farm-funded schools included 10 schools that 
ranged from preschool to high school. These schools could implement service-learning in any subject 
area. Learn and Serve-funded schools included nine middle schools, all of which were designated as Title 
I schools (high poverty) during this program. In addition, service-learning projects in the Learn and 
Serve-funded schools required a STEM (Science, Technology, Engineering and Mathematics) focus. 


Exhibit 1: Schools of Success Network Participating Schools, by Funder 


state Farm Funded 
(service-learning in any subject area) 

Learn and Serve Funded 
(STEM-focused service-learning) 

duPont Manual High 
School 

Louisville, KY 

9-12 Magnet High 
School 

Christian County 
Middle School 

Hopkinsville, KY 

6-8 Middle School 

Grant's Lick 
Elementary School 

Alexandria, 

KY 

K-5 Elementary 
School 

Detroit Edison 
Public School 
Academy 

Detroit, Ml 

P-10 Public Charter 
School (only grades 
6-8 participated) 

Greendale Middle 
School 

Greendale, 

Wl 

6-8 Middle School 

Hopkinsville 
Middle School 

Hopkinsville, KY 

6-8 Middle School 

Liberty High School 

Louisville, KY 

9-12 Alternative 
High School 

MS 442 

Brooklyn, NY 

6-8 Middle School 

Malcolm Shabazz City 
High School 

Madison, Wl 

9-12 Alternative 
High School 

New Foundations 
Charter School 

Philadelphia, PA 

P-10 Public Charter 
School (only grades 
6-8 participated) 

Montpelier High 
School 

Montpelier, 

VT 

9-12 High School 

North Drive 
Middle School 

Hopkinsville, KY 

6-8 Middle School 

Park Forest 
Elementary School 

State 

College, PA 

K-5 Elementary 
School 

School for Global 
Leaders 

New York, NY 

6-8 Middle School 

Patriot Academy 

Madison, AL 

9-12 Alternative 
High School 

Sutter Middle 
School 

Fowler, CA 

6-8 Middle School 

Raymond School 
District 

Franksville, 

Wl 

K-8 School 

Tupelo Middle 
School 

Tupelo, MS 

6-8 Middle School 

Waterford High School 

Waterford, 

CT 

9-12 High School 
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Research Methods 


The NCLCE contracted with RMC Research Denver^ to examine the Schools of Success program's effects 
on schools, community conditions and students' academic and civic engagement. While the evaluation 
was wide ranging, this issue brief presents a subset of attendance and academic performance data, as 
well as data gathered through surveys completed by students who took part in service-learning activities 
and comparison students. The results reported below are based on data collected for the 2009-10, 
2010-11 and 2011-12 school years. 

Initial analysis of data began with conversion of reading and mathematics achievement data within 
schools and within grade levels to z-scores.'* The evaluation team tested whether students who 
participated in service-learning showed higher attendance levels and more growth in reading and 
mathematics than comparison students through the use of independent groups t-tests.^ The evaluation 
team then conducted additional analyses to examine other relationships between student survey 
outcome data (academic engagement, school attachment and 21st Century skills) and achievement 
data. 

Attendance 

Overall, with a few exceptions, students who participated in service-learning did not have higher 
attendance levels than comparison students who did not participate in service-learning. We show two 
such instances in Exhibit 2. In both instances, although the attendance rates of students who 
participated in service-learning were only slightly higher than comparison students, this difference was 
statistically significant. 


Exhibit 2: Average Attendance by Group, 2010-11 Learn and Serve Grades 6-8 and 2011-12 State Farm 
Grades 6-12. 




Service-Learning 


Comparison 


Grade Level 
Year, Funder 

N 

Mean 

Attendance 

N 

Mean 

Attendance 

Significance 

Grades 6-8 
2010-11, 
Learn & Serve 

292 

.96 

44 

.93 

.001*** 

Grades 6-12 
2011-12, 
State Farm 

135 

.96 

25 

.90 

.001*** 


*** p < .001 


As was the case for other analyses of Schools of Success data, RMC used student ratings of service- 
learning quality to separate the participating students into a group that participated in "higher-quality" 
service-learning and a group that participated in "lower-quality" service-learning.® As shown in Exhibit 3, 
in two instances students who participated in higher-quality service-learning had attendance rates that 
were higher than their peers who participated in lower-quality service-learning. While one of these 
instances only approached statistical significance at the p < .05 level, the other was at a high level of 
statistical significance. 
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Exhibit 3: Average Attendance by Higher-Quality and Lower-Quality Groups, 2010-11 State Farm Grades 
3-5 and 2011-12 State Farm Grades 6-12. 




Higher-Quality 


Lower Quality 


Grade Level 
Year, Funder 

N 

Mean 

Attendance 

N 

Mean 

Attendance 

Significance 

Grades 3-5 
2010-11, 
State Farm 

111 

.972 

149 

.965 

.060 

Grades 6-12 
2011-12, 
State Farm 

135 

.96 

25 

.90 

.001*** 


*** p< .001 


As a final step in their data analysis, evaluators from RMC Research compared the attendance rates of 
participating students from Learn and Serve-funded schools across time. In this instance, evaluators 
compared students' attendance rates from 2010-11, prior to their participation in service-learning, with 
their attendance rates from 2011-12, the year in which they participated in service-learning. Though 
students' attendance rates increased only slightly from 2010-11 to 2011-12, this increase met the 
threshold for a high level of statistical significance. 


Exhibit 4: Average Attendance Over Time, 2010-11 and 2011-12, Learn and Serve Grades 6-8. 




2010-2011 


2011-2012 


Grade Level 
Funder 

N 

Mean 

Attendance 

N 

Mean 

Attendance 

Significance 

Grades 6-8 
Learn & Serve 

429 

.95 

429 

.96 

.007** 


**p<.01 


Academic Performance 

Overall, students who participated in service-learning did not exhibit higher levels of academic 
performance than comparison students. Analysis showed three instances, however, that suggest high- 
quality service-learning may have a positive relationship with students' academic performance, 
particularly in reading. First, as shown in Exhibit 5, during the 2010-11 school year, students in Learn and 
Serve-funded schools who participated in higher-quality service-learning had levels of reading 
proficiency that were higher than reading proficiency scores for students that participated in lower- 
quality service-learning at statistically significant levels. 


Exhibit 5: Average Reading Proficiency by Higher-Quality and Lower-Quality Groups, 2010-11 Learn and 
Serve Grades 6-8. 



Higher-Quality Service-Learning 

Lower-Quality Service-Learning 


Grade Level 

N 

Mean Reading 

N 

Mean Reading 

Significance 

Year, Funder 


Proficiency 


Proficiency 


Grades 6-8 

66 

3.53 

36 

3.25 

.042* 

2010-11, 






Learn & Serve 







*p < .05 
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Second, students in Learn and Serve-funded schools showed higher levels of reading achievement in 
2011-12, when they were participating in service-learning, than in 2010-11, prior to their involvement in 
service-learning. As shown in Exhibit 5, this difference was at a high level of statistical significance. 


Exhibit 6: Average Reading Proficiency Over Time, 2010-11 and 2011-12, Learn and Serve Grades 6-8. 



2010-11 


2011-12 



Grade Level 
Funder 

N 

Mean Reading 
Proficiency 

N 

Mean Reading 
Proficiency 

Significance 

Grades 6-8 
Learn & Serve 

429 

2.52 

472 

2.73 

.005** 


** p<.01 


Finally, data from State Farm-funded schools in 2009-10 suggests that service-learning may help to 
improve levels of school attachment and 21st century skills acquisition for students who are at the lower 
end of the academic performance spectrum. When the research team grouped students into four equal 
quartiles, the largest consistent and significant difference between service-learning and comparison 
groups were found for those students with the lowest reading scores. As shown in Exhibit 6, students in 
service-learning who had the lowest reading scores showed gains in their ratings of school attachment 
and acquisition of 21st century skills, while students in the comparison group showed decreases in these 
same areas. 


Exhibit 7: Gains in Outcome Measures for Students with the Lowest Scores in Reading, 2009-10, State 
Farm Grades 3-5. 

0.4 


0.2 


Gains -0.2 


-0.4 


- 0.6 


- 0.8 


0.29 


0.19 



School Attachment 


21st Cenljjry Skil 


-0.62 


I Service-Learning 
I Comparison 


-0.74 


Conclusions and Next Steps 

The evaluation findings presented in this analysis do not show that service-learning has a clear, positive 
and statistically significant relationship with students' school attendance and academic performance. 
This (non) finding may be for two reasons. 
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First, we saw much variation in the quality of service-learning implementation across sites. Service- 
learning is a complex pedagogical approach that helps to guide instruction, not a standardized, one-size- 
fits-all, scripted instructional method. Teacher judgment and expertise, as well as local resources, 
constraints and conditions all play a role in how well teachers are able to align their implementation of 
service-learning with quality standards. Because many of the teachers in participating schools were 
relatively new to service-learning, the service-learning these teachers implemented was of mixed 
quality. Rather than being indicative of the failures of service-learning to impact student outcomes, this 
finding likely points to the steep learning curve for teachers that accompanies any complex pedagogy, 
including service-learning. 

Second, a ceiling effect likely limited any observable change in attendance rates. Average attendance 
rates for all groups and years were 90 percent, at a minimum, and most hovered around 95 percent. 
Because the schools in the program already had high attendance rates, the amount of improvement 
possible was rather limited. The fact that we found any statistically significant findings for attendance 
rates suggest that service-learning may have a positive impact on students' school attendance. 

The findings of this evaluation do suggest, however, that a positive relationship between service- 
learning and school attendance and academic performance might exist (1) when service-learning meets 
standards for high-quality practice and (2) for the lowest-achieving students. When RMC divided site- 
level implementation of service-learning into "higher quality" and "lower quality," those students who 
indicated that they participated in higher-quality service-learning showed higher levels of attendance 
and reading achievement at higher levels of statistical significance on most key outcomes than did 
students who participated in lower-quality service-learning. 

Because RMC used the National Youth Leadership Council's K-12 Service-Learning Standards for Quality 
Practice^ as the basis for its construction of student survey items related to assessment of quality, these 
results suggest that teachers, administrators and policymakers need to pay close attention to the quality 
standards when creating and implementing service-learning programs. These standards appear to offer 
critical guidance in the "how to" of service-learning, particularly when student outcomes are a key 
concern. With regard to the lowest-achieving students, the findings here suggest some positive impact 
for service-learning, though further research is necessary to confirm and fully capture the nature and 
breadth of this impact. 

The benefits from service-learning may appear to be minor given the small increases service-learning 
students gained on the outcome measures considered in this brief. In interpreting these apparently 
small increases, consideration of the length of the evaluation period (one academic year, or 
approximately nine months) and how increases may accumulate over time is necessary. For example, a 
student who participates in several consecutive school years of service-learning may very well 
accumulate these increases over time (e.g., a .12 gain for a single year may be a much larger gain over 
four years). As a next step in this line of research, longitudinal studies will be necessary to confirm 
whether or not this is the case. In addition, analysis of aggregated data from across similar service- 
learning programs is necessary to further solidify the findings of NCLCE's own evaluations. 
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Endnotes 


^ The Schools of Success network was part of a larger evaluation study conducted by RMC Research Denver that used a set of 
common measures across a cluster of Learn and Serve states (Arizona, Hawaii, Louisiana, Michigan, Minnesota, Missouri, Ohio 
and Wisconsin) and national programs (e.g.. Youth Service America's STEMester of Service). 

^ Terry Pickeral, Tiffani Lennon and Jennifer Piscatelli, Service-Learning Policies and Practices: A Research-Based Advocacy 
Paper. 2008 (Denver: Education Commission of the States). 

^ RMC also conducted evaluations of other states and national programs in our Learn and Serve cluster. 

^ The z-score transformation standardizes variables to the same scale, producing new variables with a mean of 0 and a standard 
deviation of 1. 

^ A f test is a statistical procedure that is commonly used to examine differences in mean values over time or across two groups. 
® A median split was conducted on the student-rated service-learning program quality subscale. Two categories were created 
from the split (higher-quality and lower-quality programs). 

^ K-12 Service-Learning Standards for Quality Practice. 2008 (Saint Paul: National Youth Leadership Council). 
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